A researcher built a fictional billing database and had five AI models create semantic layers (business logic definitions) for it, then tested how well those layers helped the models answer finance questions. The results show that a well-built layer can dramatically improve accuracy, but models often make confident mistakes about scope and conventions that human finance teams would catch.

• Haiku failed to build a usable layer at all; the four models that succeeded got the math right but disagreed on scope—which accounts count as customers, whether subsidiaries roll up to parents, and how to handle QA test data.
• A reference layer written by a human improved Opus from 17 of 30 correct answers to 30 of 30, while a flawed layer built by Sonnet dropped Fable from 18 correct to 12, showing that semantic layers can either unlock or degrade model performance.
• Models built layers with high confidence regardless of accuracy: Muse matched finance definitions on all but one rule, while Opus and Fable made the same incorrect assumption about subsidiary rollups that finance never used, and Sonnet inconsistently filtered QA accounts across different metrics.